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A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH{S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 
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DETAILED ACTION 



Di^wings 

1 . The drawings were received on 05 February 2004. These drawings are 
acceptable. 



Double Patenting 

The nonstatutory double patenting rejection is based on a judicially created 
doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the 
unjustified or improper timewise extension of the "right to exclude" granted by a patent 
and to prevent possible harassment by multiple assignees. See In re Goodman, 1 1 
F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 
USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 
1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970);and, In re Thorington, 
41 8 F.2d 528, 1 63 USPQ 644 (CCPA 1 969). 

A timely filed terminal disclaimer in compliance with 37 CFR 1 .321(c) may be 
used to overcome an actual or provisional rejection based on a nonstatutory double 
patenting ground provided the conflicting application or patent is shown to be commonly 
owned with this application. See 37 CFR 1 .130(b). 

Effective January 1 , 1 994, a registered attorney or agent of record may sign a 
terminal disclaimer. A terminal disclaimer signed by the assignee must fully comply with 
37 CFR 3.73(b). 

2. Claims 23-25 and 32-33 are rejected under the judicially aeated doctrine of 
obviousness-type double patenting as being unpatentable over claim 9 of U.S. Patent 
No. 6,629,107 in view of U.S. Patent No. 5,894,306 to Ichimura. 

All limitations of pending claims 23-25 and 32-33 are included in the set of 
limitations making up claim 9 of the '107 Patent, with the exception of "a microphone to 
input speech" and the information recognition unit configured "to identify a speaker from 



the speech." Ichimura discloses a multimedia information collection control apparatus 
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similar to claim 9 of the '107 Patent, and further comprising a microphone to input 
speech [15] and a recognition unit [24] configured to identify a speaker from the speech 
[See Column 7, lines 48-51], and to identify a speaker from the speech ['a speaker is 
recognized according to the extraction of features of the input audio signals' (Column 8, 
lines 13-20)] as claimed. It would have been obvious to one of ordinary skill in the art at 
the time the invention was made to add Ichimura's microphone and speaker recognition 
functionality to the apparatus of claim 9 of the '107 Patent to obtain the invention as 
recited in pending claims 23-25 and 32-33. One would have been motivated to do so in 
order to recognize and associate the text of words spoken by a particular speaker with 
the multimedia information associated with that speaker, as disclosed by Ichimura. 

3. Claims 34-36 are rejected under the judicially created doctrine of obviousness- 
type double patenting as being unpatentable over claims 3-5 (respectively) of U.S. 
Patent No. 6,629,107 in view of U.S. Patent No. 5,894,306 to Ichimura and U.S. Patent 
Application Publication No. 2002/0085759 to Davies et al. 

All limitations of pending claims 34-36 are included in the sets of limitations 
making up claims 3-5 of the '107 Patent respectively, with the exception of "a 
microphone to input speech" and the information recognition unit configured "to 
recognize characters in the image, to extract a person's name from the recognition 
result of the characters, and to identify a speaker from the speech." 

Ichimura discloses a multimedia information collection control apparatus similar 
to the claims of the '107 Patent, and further comprising a microphone to input speech 
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[15] and a recognition unit [24] configured to identify a speaker from the speech [See 
Column 7, lines 48-51], and to identify a speaker from the speech ['a speaker is 
recognized according to the extraction of features of the input audio signals' (Column 8, 
lines 13-20)] as claimed. Davies also discloses a multimedia information collection 
control apparatus similar to the claims of the '107 Patent, and further comprising an 
information recognition unit configured to recognize characters in an image and to 
extract a person's name from the recognition result [See paragraph 0014] as claimed. 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to add Ichimura's microphone and speaker recognition functionality 
and Davies character and person's name recognition functionality to the apparatus of 
claims 3-5 of the '107 Patent to obtain the invention as recited in pending claims 34-36 
respectively. One would have been motivated to do so in order to recognize and 
associate the text of words spoken by a particular speaker with the multimedia 
information associated with that speaker, as disclosed by Ichimura, and to register 
names of new users easily and without manual input, as disclosed by Davies. 

Claim Rejections - 35 (JSC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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4. This application currently names joint inventors. In considering patentability of 
the claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of 
the various claims was commonly owned at the time any inventions covered therein 
were made absent any evidence to the contrary. Applicant is advised of the obligation 
under 37 CFR 1 .56 to point out the inventor and invention dates of each claim that was 
not commonly owned at the time a later invention was made in order for the examiner to 
consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 102(e), (f) or (g) 
prior art under 35 U.S.C. 103(a). 

5. Claims 23-24 and 37-42 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over U.S. Patent No. 5,894,306 to Ichimura in view of U.S. Patent 
Application Publication No. 2002/0085759 to Davies et al. and U.S. Patent No. 
6,363,380 to Dimitrova. 

Referring to claim 23, Ichimura discloses a system and method for collecting and 
archiving multimedia information substantially as claimed. See Figures 1-12 and the 
corresponding portions of the specification for this disclosure. Ichimura discloses a 
multimedia information collection control apparatus, comprising: 

a multimedia information collection unit [Electronic Meeting Device 10] configured 
to collect multimedia information ['image data', 'audio data' and 'user input from 
electronic pen 1 3'] from a plurality of kinds of input devices [Video Camera 16 (image), 
Microphone 15 (audio), and Touch Panel/Tablet 12 (pen input)]; 

a multimedia correspondence memory [First Memory Section 22] configured to 
correspondingly store the multimedia information including an image and a speech ['The 
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audio signal from microphone 15 and the image signal from video camera 16 are 
sequentially stored into the first memory section 22' (Column 8, line 62 et seq.)]; 

an information recognition unit [State Detector 24] configured to recognize 
characters in the image [See Column 7, lines 48-51], and to identify a speaker from the 
speech ['a speaker is recognized according to the extraction of features of the input 
audio signals' (Column 8, lines 13-20) and 'the method used to identify a speaker... can 
involve identification based on the facial and mouth motions of the speakers determined 
from the image data' (Column 15, lines 5-20)]; and 

a second memory section [23] configured to store relational pointers to the 
multimedia information [stored in the first memory section] in correspondence with the 
identified speaker by using the person's name ['the name of the recognized speaker is 
transferred to the second memory section 21 as a result of detection' (Column 8, lines 
13-20) also see Figures 5-12 and the corresponding portions of the specification for the 
disclosure of the analysis]. 

Ichimura does not explicitly extract a person's name from the recognition result of 
the character's as claimed. However, it is clear that a name handwritten on the tablet 
(12) would be recognized by the character recognition unit just as any other handwritten 
text would be. Thus, one could easily infer that the person's name could be extracted 
from its character recognition. Further, Davies discloses a system and method similar 
to that of Ichimura, wherein a person's name is extracted from optical character 
recognition of an image (e.g. of a business card) to identify a person, and then data is 
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stored in correspondence with the person's name within a database. See paragraph 
0014 for the details of this disclosure. 

Neither Ichimura nor Davies explicitly discloses "a multimedia database" 
configured to relationally store the multimedia information in correspondence with the 
identified speaker as claimed. However, Ichimura's second memory section 23 does 
store this relationship as described above and shown in Figure 12. Dimitrova discloses 
a system and method for gathering multimedia data and storing it with meaningful 
relationships between the different types of data very similar to that of Ichimura. See ' 
Figures 1-6B and the corresponding portions of the specification for this disclosure. 
Dimitrova's event relationships are stored as 'video stories' in a video story database 
(multimedia database). Refer specifically to column 6, line 60 - column 7, line 55 and 
column 10, line 20 - column 13, line 17 for the details of this disclosure. Thus, 
Dimitrova teaches a multimedia database configured to relationally store the multimedia 
information as story data analyzed by the system. 

Given the desire for increased memory space to store multimedia data for 
meetings that last an extensive amount of time, the desire for access to a larger archive 
of multimedia data for many meetings, and the equivalent functionality of the systems, it 
would have been obvious to one of ordinary skill in the art at the time the invention was 
made to swap Ichimura's Second Memory Section 23 with a multimedia database, such 
as that of Dimitrova, to store the multimedia information (image data, audio data, and 
handwritten user input data) relationally in correspondence with the person's name 
analyzed by Ichimura's state detector 24. Further, it would have been obvious to one of 
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ordinary sl^ill in the art at the time the invention was made to incorporate Davies' 
functionality of extracting of a person's name by optical character recognition into the 
system and method of Ichimura v. Dimitrova to obtain the invention as claimed. One 
would have been motivated to do so in order to identify and store information 
corresponding to a new user of the system without necessity of manual input, as 
described by Davies. 

Referring to claim 24, Ichimura v. Davies and Dimitrova teaches the multimedia 
information collection control apparatus of claim 23, as above, wherein the plurality of 
kinds of input devices include a camera to input the image [Ichimura: 16], and a 
microphone to input the speech [Ichimura; 15] as claimed. 

Referring to claim 37, Ichimura v. Davies and Dimitrova teaches the apparatus of 
claim 23, as above, wherein said multimedia database correspondingly stores [See 
claim 23 above] a record identification [Ichimura: Participant ID - user input ID] of each 
item of the multimedia information as claimed. 

Referring to claim 38, Ichimura v. Davies and Dimitrova teaches the apparatus of 
claim 37, as above, wherein said multimedia information collection unit [Ichimura: See 
Figs. 1-2] includes an information addition unit [Ichimura: 10] configured to additionally 
input information to said multimedia database [See claim 23 above]; and wherein said 
multimedia database additionally stores the input information [See claim 23 above]... as 
claimed. 

Referring to claim 39, Ichimura v. Davies and Dimitrova teaches the apparatus of 
claim 37, as above, further comprising a dialogue control unit [Ichimura: Playback 
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Specifying Section 27 (column 12, line 57 - column 14, line 42)] configured to input a 
retrieval request from a user [(See column 13, lines 8-20)], to analyze the retrieval 
request [(See column 13, lines 26-37)], and to generate a retrieval key ['the X-Y 
coordinates groups' (Again, see column 1 3, lines 26-37)] according to the analysis 
result as claimed. 

Referring to claim 40, Ichimura v. Davies and Dimitrova teaches the apparatus of 
claim 39, as above, further comprising a retrieval control unit [Ichimura: Control Section 
30 in conjunction with Playback Specifying Section 27] configured to retrieve the 
multimedia database by comparing the retrieval key with the record identification of 
each item of the multimedia information [(See column 13, lines 30-48)]; and wherein 
said multimedia information collection unit [Electronic Meeting Device 10] presents the 
retrieved information through the display [playback of image data (See Figure 2)] as 
claimed. 

Claims 41 and 42 are rejected on the same basis as claim 23. See the 
discussion regarding claim 23 above for the details of this disclosure. 

6. Claims 25-31 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Ichimura in view of Davies and Dimitrova as applied to claims 23-24 above, and further 
in view of U.S. Patent No. 5,761,340 to Suzuki. 

Referring to claim 25, Ichimura v. Davies and Dimitrova as applied to claim 23 
above teaches the apparatus of claim 24, as above, wherein said multimedia 
information collection unit [Ichimura: Electronic Meeting Device 10] includes a display 
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[Screen 1 1] to output the image [(See column 7, lines 13-20)], and an indicator 
[Electronic Pen 13 in conjunction with Touch Panel/T ablet 12] to artificially indicate 
marks, figures, or handwritten text [(See column 7, lines 43-60)] on the display. 

Ichimura does not explicitly disclose that a purpose of the electronic pen indicator 
is to "artificially specify a recognition area of the image" as claimed. However, Ichimura 
does disclose that the pen could be used for "editing data indicating 
moving/copying/cutting of displayed user-input data." (Column 7, lines 55-56) Ichimura 
also teaches that the system can enclose certain image data in a clipping window 
(CLW) as shown in Figure 8. These two disclosures provide direct suggestion for using 
the electronic pen indicator to indicate a mark representing a desired area of the 
displayed image to which some sort of operation should be performed. 

Suzuki discloses a pen type input device very similar to Ichimura's Electronic Pen 
13 and Touch Panel/Tablet 12. See Figures 1-9 and the corresponding portions of the 
specification for this disclosure. Suzuki's system performs character recognition, other 
types of image recognition, and editing operations on image data enclosed within a 
target area specified as a mark on the input/display device 1 3. Refer specifically to 
Figures 1 , 3A-3D, 5, 7A-7D & 9 and the corresponding portions of the specification 
(Particularly column 5, lines 29-33) for the details of this disclosure. Thus, Suzuki 
teaches artificially indicating a mark representing a recognition area as a user's desired 
area of the image as claimed. 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to incorporate Suzuki's functionality of indicating a mark 
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representing a recognition area (or editing area) as a user's desired area of the image 
into Ichimura's (as modified by Davies and Dimitrova) electronic meeting device, such 
that a user could mark a specific area of a displayed image for recognition. One would 
have been motivated to do so because of the suggestion provided by Ichimura above, 
considering that the primary operations of Ichimura's system are recognition. 

Referring to claim 26, Ichimura v. Davies, Dimitrova and Suzuki teaches the 
apparatus of claim 25, as above, v\/herein said display presents an attribute selection 
window [Suzuki: See Figs. 7-11; Davies: See Figs. 4-5] including a plurality of attribute 
items [Suzuki: edit commands; Davies: 'stickers']; and wherein said Information 
recognition unit prepares a plurality of knowledge dictionaries [Davies: services (See 
414 & 516-518)] each corresponding to each of the plurality of attribute Items [Davies: a 
specific service is associated with each 'sticker' (attribute)] as claimed. 

Referring to claim 27, Ichimura v. Davies, Dimitrova and Suzuki teaches the 
apparatus of claim 26, as above, wherein said indicator artificially selects one attribute 
[Suzuki: based on the mark] corresponding to the characters in the image [Davies: 
sticker (See 414 & 516-51 8)]... as claimed. 

Referring to claims 28 and 29, Ichimura v. Davies, Dimitrova and Suzuki teaches 
the apparatus of claim 27, as above, wherein said information recognition unit 
recognizes the characters in the image by using one of the plurality of knowledge 
dictionaries [See Davies' place in combination as in claims 23 & 26-27 above] 
corresponding to the one attribute item [See above]; wherein if the one attribute item is 
a card [Davies: business card (See paragraph 0014)], said multimedia database stores 
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the recognition result of the characters of the card as the speaker's personal data [See 
claim 23 above] as claimed. 

Referring to claim 30, Ichimura v. Davies, Dimitrova and Suzuki teaches the 
apparatus of claim 29, as above, wherein said information recognition unit [Ichimura: 
state detector 24] includes a speech recognition unit [Speaker Recognition from audio 
data (See Figures 5 & 6 and the corresponding portions of the specification)] configured 
to recognize the speech in the multimedia information; and wherein said multimedia 
database [See claim 23 above] stores the recognition result of the speech [Ichimura: 
See Figure 12 & claim 23 above]... as claimed. 

Referring to claim 31 , Ichimura v. Davies, Dimitrova and Suzuki teaches the 
apparatus of claim 29, as above, wherein said information recognition unit [Ichimura: 
state detector 24] includes a face recognition unit [Speaker Recognition from image 
data (See Column 1 5, lines 5-20)] configured to recognize a facial characteristic of a 
face area in the image; and wherein said multimedia database [See claim 23 above] 
stores the recognition result of the facial characteristic [Ichimura; See Figure 12 & claim 
23 above]... as claimed. 

Conclusion 

7. The prior art made of record and not relied upon is considered pertinent to 
applicants' disclosure. Specifically, the Patent references made of record but not relied 
upon are considered pertinent to portions of applicants' claims. 
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8. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Brian Goddard whose telephone number is 571-272- 
4020. The examiner can normally be reached on M-F, 9 AM - 5 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Safet Metjahic can be reached on 571-272-4023. The fax phone number for 
the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 

bdg 

28 April 2005 
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